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Use of Taguchi Design of Experiments 
to Determine ALPLS Ascent Delta-V Sensitivities and 
Total Mass Sensitivities to Release Conditions and Vehicle Parameters 


ABSTRACT 


NASA is currently assessing the engineering feasibility, safety and reliability, and the 
infrastructure and operational requirements of an of an air launched personnel launch 
system (ALPLS) for transportation of personnel to low earth orbit. One of the Study 
Requirements calls for the determination of ALPLS ascent delta-V sensitivities to release 
conditions (Altitude, Mach, Flight Path Angle, Delay Time) and vehicle parameters (Isp, 
Ignition Tvac/W, L/D). 

Previous parametric studies of this type have been accomplished by varying and optimizing 
one parameter at a time. Parameter interactions are considered by re-evaluating sensitive 
parameters while holding other variables at or near previously determined optimal values. 
A traditional, scientific, approach results in the experimenter methodically evaluating every 
possible combination of the parameters. Note that if an experiment involved seven 
parameters, each with only two possible levels, a total of 2 7 or 128 experiments would have 
to be conducted. Statisticians, as advocated by Taguchi, have developed more efficient test 
plans, which are referred to as fractional factorial experiments (FFEs). Certain treatment 
conditions are chosen to maintain orthogonality among the various factors and interactions. 
Taguchi design of experiments methods can be used to more efficiently analyze the optimal 
values of the parameters indicated above. By using Fractional Experiment Designs 
(Orthogonal Arrays) the number of runs can be significantly reduced with a better 
evaluation of the parameter interactions. Orthogonal experiment design is a time-saving 
testing strategy that draws on orthogonal arrays to pinpoint areas where variation may be 
successfully reduced. 

The objective of this study is to evaluate the use of Taguchi's Design of Experiment 
Methods to improve the effectiveness of this and future parametric studies. Taguchi 
Methods will be applied in addition to the typical approach to provide a mechanism for 
comparing the results and the cost or effort necessary to complete the studies. It is 
anticipated that results of this study should include an improved systematic analysis process, 
an increase in information obtained at a lower cost, and a more robust, cost effective vehicle 
design. 
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INTRODUCTION 


The objective of this study is the demonstration of an improved methodology for the 
Parametric Design Phase of the space vehicle design process utilizing Taguchi Design of 
Experiment Methods. Although the design of experiment methods such as fractional 
orthogonal arrays are in common use in experimental settings, they are not being utilized 
in areas where computer models are available. Computer analysis of complex models can 
be more efficiently accomplished if computer runs are viewed and planned as experiments. 
Standardizing the evaluation process using design of experiment methods eliminates many 
of the common solution approaches such as random trial and error, the varying of one 
variable at a time, or the methodical and time consuming evaluation of every possible 
combination of variable levels. The use of experimental design should result in better 
designs using fewer computer runs while obtaining an interpretable evaluation of the designs 
sensitivity to their environment or more robust designs. 

OVERVIEW OF PARAMETER DESIGN AND TAGUCHI METHODS 

During product development, engineers and scientists are typically faced with two opposing 
requirements; improve or optimize performance and reduce or minimize cost. The process 
of searching for factors or parameters affecting performance and/or cost is usually 
experimental in nature. After a set of relevant parameters are assembled, the experimenter 
is faced with the task of determining which combination of parameter values achieve the 
desired results. It is the quality of this decision that can be improved when proper tests 
strategies are used. The most common test plan is to evaluate and optimize one parameter 
at a time. If there happens to be an interaction of the factor being studied with any other 
factor, the interaction will not be observed as all other factors are being held constant. This 
loss of interaction information is accepted as unavoidable as the only other perceived 
alternative is to perform a full factorial experiment, testing every possible combination of 
factors. This is usually not feasible as most real problems involve many parameters with 
three or more possible levels requiring thousands of experiment runs. Taguchi is an 
advocate of more efficient test plans, which are referred to as fractional factorial 
experiments (FFEs). FFEs use only a portion of the total possible combinations to estimate 
the main factor effects and some, not all, of the interactions (1) . FFEs are balanced 
experiments developed using orthogonal arrays. They can be used to evaluate many 
parameters with a minimum number of tests. 

As suggested above, the goal of parameter design is to determine the parameter values of 
a product or process so that the product is functional, exhibits a high level of performance 
and is minimally sensitive to noise. The strategy is to design a high quality product which 
can be produced from low grade, low cost components with broad tolerances. This 
improved quality and reduced variability is achieved by selecting the optimum parameter 
values so that the product is least sensitive to input and noise variations. Conventional 
quality improvement techniques reduce product variation by removing the cause which is 
usually expensive, while Taguchi methods reduce variation by becoming less sensitive to 
input variations without removing the cause of variation. Improved quality is, therefore, 
achieved without or with minimal increase in cost. 
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Taguchi Methods take advantage of non-linear effects and the interaction between control 
factors and noise factors in order to obtain designs that are more "robust". Taguchi’s 
approach to design of experiments utilizes techniques that are cost effective and directly 
applicable to the problems and requirements of modern industry. Although design of 
experiments has been around for some time, it is seldom used for product development. 

A parameter design experiment typically involves two types of factors: 

Control Factors whose levels can be set and maintained. 

Noise Factors whose level either cannot or will not be set or maintained, yet 
which could affect the performance of the functional characteristics 
Parameter design examines interactions between control factors and noise factors in order 
to achieve robustness. It is a search for parameter levels at which a characteristic is stable, 
despite the use of inexpensive components and materials or external conditions. 

The major steps in designing, conducting, and analyzing an experiment are as follows: 

1. Selection of factors and/or interactions to be evaluated 

2. Selection of number of levels for the factors 

3. Selection of appropriate Orthogonal Array 

4. Assignment of factors and/or interactions to columns 

5. Performance of experiments 

6. Analysis of results 

7. Performance of confirmation experiment. 

The goal of the experiment is to determine the combination of parameters that gives the 
most stable and reliable performance at the lowest cost. 

PROBLEM STATEMENT 

The Systems Definition Branch, Systems Engineering Division, is currently assessing the 
engineering feasibility, safety and reliability, and the infrastructure and operational 
requirements of an Air Launched Personnel Launch System (Air Launched PLS) for 
transportation of personnel to low earth orbit. One of the study requirements calls for the 
determination of the ascent delta-V sensitivities to release conditions (Altitude, Mach, Flight 
Path Angle, Delay time) and vehicle parameters (Isp, Ignition Tvac/W, L/D). 

Previous parametric studies of this type have been accomplished by varying and optimizing 
one parameter at a time. Parameter interactions are considered by re-evaluating sensitive 
parameters while holding other variables at or near previously determined optimal values. 
This approach, while time consuming, is preferred by analysts as it provides considerable 
insight regarding the sensitivities and costs associated with deviations from optimal 
conditions. This study will evaluate the use of Taguchi Design of Experiment Methods for 
space vehicle design parametric studies by performing an analysis of the Air Launched PLS 
in conjunction with the traditional approach to provide a mechanism for comparing the 
results and the effort necessary to complete the studies. 
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EXPERIMENTAL DESIGN AND ANALYSIS 


Referring to the Air Launched Personnel Launch System study statement of work, section 
IIIB called for the following: 

"Determine ALPLS ascent delta-V sensitivities to release conditions (Altitude, 

Mach, Flight Path Angle, Delay Time) and vehicle parameters (Isp, Ignition 
Tvac/W, L/D). Sensitivities are to be evaluated over the following parameter 
ranges using a two-stage vehicle analysis: 


- Altitude: 

- Mach: 

- Gamma: 

- Delay Time: 


25K to 45 K ft. 

0.7 (or based upon aircraft performance) 
0 to 30 deg. 

0 to 6 sec. 


- Isp: Based upon selected propulsion systems 

- Ignition Tvac/W: 1.3 to 1.9 (Ballistic 

0.9 to 1.9 (Aero Assisted) 

- L/D: Ballistic, low, medium, high aero assist" 


The process of designing, conducting, and analyzing the parametric study directly followed 
the steps given earlier. 


Selection of Parameters and Levels to be Evaluated 

The number of parameters given above was reduced to sue with the conclusion that the 
aircraft would be flown at maximum velocity for the release of the PLS. The parameters 
and the levels selected for the Taguchi Study were as follows: 


Variable Levels 


Altitude: 

25,000 ft. 

35,000 ft. 

45,000 ft. 

Gamma: 

0 degrees 

10 degrees 

20 degrees 

Delay Time: 

2 seconds 

4 seconds 

6 seconds 

Isp: 

LO x /LH 2 

A50/N,O 4 

HTBP 

Ign Tvac/W 

1.5 

1.7 

1.9 

L/D 

Ballistic 

Winged 



The Flight Path Angle (gamma) was varied in 10 degree increments to conform with the 
increment size being used in the traditional approach. This limited the range to a total of 
20 degrees in order to limit the number of levels in the Taguchi study to three. The Lift 
to Drag ratio (L/D) was reduced to two levels with project teams decision to model only 
one aero-assist version having wings sized to maintain a horizontal flight path and facilitate 
separation from the aircraft at the time of release. 


1-5 



Selection of Appropriate Orthogonal Array 

The selection of an orthogonal array depends on the number of factors and interactions to 
be modeled and the number of levels for the factors. Initially, little was known about the 
relevant interactions; however, it was believed that the engine type (Isp) and L/D 
parameters were the dominant factors. The L18 orthogonal array, shown in Table I, was 
selected for the study as it can accommodate two and three level parameters and the 
interactions between the three level columns are distributed more or less uniformly to all 
of the other three-level columns (2). 

Assignment of Factors to Columns 

Table II contains the parameter column assignments. The Lift to Drag ratio was assigned 
to column one as it was the only two level parameter while the Isp was assigned to column 
two to enable the modeling of the interactions between the two probable dominant 
variables. It was believed that the prevalent interaction would occur between these two 
parameters due to the significant differences in the magnitude of the Isps. These column 
assignments also facilitated the setup for the various trials as different computer models 
were needed when switching from a ballistic vehicle to a winged vehicle and different rocket 
sizing models were needed for each different engine type. Columns seven and eight 
remained unassigned providing a measure of error. It should be pointed out at this time 
that the selection of the L18 orthogonal array was not appropriate for this study as the main 
effects of the first two columns and their interaction dominated the experiment. This can 
be readily seen in the raw data, also shown in Table II. The very large masses (m glow) of 
the Winged-HTBP vehicles significantly affected the results of the study to the extent that 



Trial hcl-J 2 . 

1 i i 

2 1 1 

3 1 1 

4 1 2 

5 1 2 

6 1 2 

7 1 3 

8 1 3 

9 1 3 

10 2 1 

11 2 1 

12 2 1 

13 2 2 

14 2 2 

15 2 2 

16 2 3 



Table I. L18 Orthogonal Array 


L 18 (2 1 x3 7 ) 

3 4 5 6 Z 2. 

11 11 11 

2 2 2 2 2 2 

3 3 3 3 3 3 

1 1 2 2 3 3 

2 2 3 3 1 1 

3 3 11 2 2 

12 13 2 3 

2 3 2 1 3 1 

3 1 3 2 12 

3 1 3 2 2 1 

2 1 13 3 2 

3 2 2 1 1 3 

12 3 1 3 2 

2 3 12 13 

3 1 2 3 2 1 

1 3 2 3 1 2 

2 1 3 1 2 3 

3 2 1 2 3 1 



Note: This is a specially 

designed array. An inter- 
action is built between the 
first two columns. This 
interaction information can 
be obtained without sacri- 
ficing any other column. 
Interactions between three- 
level columns are distri- 
buted more or less 
uniformly to all the other 
three-level columns, which 
permits investigation of 
main effects. Thus, it is a 
highly recommended array 
for experiments. 
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Table II. Air Launched PLS Parameter Column Assignments 



1 

Ballistic 

LOX/LH2 

25 

0 

2 

1.5 

1 

1 

29,661 

2 

Ballistic 

LOx/LH2 

35 

10 

4 

1.7 

2 

2 

28,572 

3 

Ballistic 

LOVLH2 

45 

20 

6 

1.9 

3 

3 

27,875 

4 

Ballistic 

A5CVN204 

25 

0 

4 

1.7 

3 

3 

29,690 

5 

Ballistic 

A50/N2O4 

35 

10 

6 

1.9 

1 

1 

28,393 

6 

Ballistic 

A50/N2O4 

45 

20 

2 

1.5 

2 

2 

28,042 

7 

Ballistic 

HTBP 

25 

10 

2 

1.9 

2 

3 

28,417 

8 

Ballistic 

HTBP 

35 

20 

4 

1.5 

3 

1 

28,384 

9 

Ballistic 

HTBP 

45 

0 

6 

1.7 

1 

2 

28,369 

10 

Winged 

LOX/LH2 

25 

20 

6 

1.7 

2 

1 

27,929 

11 

Winged 

LOX/LH2 

35 

0 

2 

1.9 

3 

2 

27,808 

12 

Winged 

LOX/LH2 

45 

10 

4 

1.5 

1 

3 

28,309 

13 

Winged 

A50/N2O4 

25 

10 

6 

1.5 

3 

2 

28,157 

14 

Winged 

A50/N2O4 

35 

20 

2 

1.7 

1 

3 

27,777 

15 

Winged 

A50/N2O4 

45 

0 

4 

1.9 

2 

1 

27.781 

16 

Winged 

HTBP 

25 

20 

4 

1.9 

1 

2 

27,601 

17 

Winged 

HTBP 

35 

0 

6 

1.5 

2 

3 

27,839 

18 

Winged 

HTBP 

45 

10 

2 

1.7 

3 

1 

27,542 


730,965 
661 ,753 
624,690 
1,949,102 
1,654,014 
1,485,159 
2,349,550 
2,332,576 
2,324,922 
722,221 
773,352 
904,770 
1,914,820 
2,028,850 
2,467,980 
2,789,500 
3,502,700 


all of the parameter levels used in the 18th trial were found to be the worse possible settings 
in the data analysis phase although an altitude of 45,000 ft. and a delay time of two seconds 
were expected to be optimal. 

Performance of the Experiment 

The computer analysis necessary for the completion of the 18 trial runs was performed by 
the System Definition Branch of the Systems Engineering Division and their Engineering 
Support Contractor, Lockheed Engineering and Science Company. The parametric study 
using the traditional approach required approximately 90 computer program setups, each 
requiring approximately 30 computer runs for a total number of computer runs in the order 
of 2500 to 3000. The Taguchi Method required only 18 program setups with only a few 
computer runs to determine optimal delta-V splits and flight profile to minimize total 
vehicle mass at the specified initial parameter conditions. At this point the effort required 
to perform the Taguchi study is significantly less than the traditional approach; however, 
more analysis will be necessary before the Taguchi Method provides the information 
required for the Air Launched PLS study. 

Analysis of Results 

Analysis of variance is typically used to determine the extent to which factors contribute to 
variation and to test significance of factorial effects. The factors that strongly effect 
variation can, however, be determined by recognizing the differences in the average effects 
of factors. These differences in effect are clearly seen in response graphs (3). 
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The Response Graphs, shown in Figure 1, show the response of the gross lift off mass with 
respect to each of the six parameters studied. The graphs show a slight, possibly significant 
preference to the ballistic configuration and a strong preference for the LO x /OH, fueled 
engine. The remaining parameters appear to have little effect on the vehicle mass, with the 
optimal release conditions at 25,000 ft altitude, a flight path angle of 20 degrees, a drop time 
of six seconds prior to firing of the engines and an optimal thrust to weight ratio of 1.9. The 
preference for a low release altitude can be explained by the realization that the winged 
space craft requires a significantly larger wing to support the craft in the thinner atmosphere 
of the higher altitude. The preference for large drop times is caused by the insensitivity of 
the winged vehicles to drop time as the available lift minimized the loss in altitude over time 
and the dominance of the winged vehicles due to their larger masses. 

The Interaction Graphs, given in Figure 2, demonstrate the interactions between the Lift 
to Drag parameter and the other five parameters. Although the LI 8 orthogonal array is 
only designed to model the interaction between the first two columns, L/D versus Isp, 
considerable insight can be obtained from the study of all of the graphs. The L/D versus 
Isp graph shows that an interaction is present as the lines are not parallel. It should also 
be noted that both ballistic and winged vehicles preferred the LO x /OH, engines. The L/D 
versus Altitude graph shows the preference for a high altitude release for the ballistic 
vehicle and a low altitude release for the winged space vehicle. The remaining graphs are 
more severely hampered by the interactions not modeled with unexplained switches in the 
preferences of the ballistic and winged vehicles. The Interaction Graphs, given in Figure 
3, show a much more erratic behavior. The Altitude versus L/D graph again clearly shows 
the preference of a low altitude release for the winged vehicle and a high altitude release 
for the ballistic one. However, the Altitude versus Flight Path Angle, Altitude versus 
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Figure 3. Altitude Interaction Graphs 
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Drop Time, and Altitude versus Thrust to Weight Ratio graphs only demonstrate the 
significance of the interactions. The primary conclusion that can be drawn from these 
graphs is that the LI 8 orthogonal array was not suitable for this study as the interactions 
were not as "uniformly distributed" as originally anticipated. 

Another indication regarding the significance of the interactions can be seen in the analysis 
of variance, refer to Table HI. The sum of squares for the factors in columns one and two 
are clearly more significant than the remaining factors. It is estimated that L/D, Isp, and 
their interaction account for as much as 90% of the total observed variation. It should be 
noted, however, that an analysis of variance is not valid, as an estimate of experiment error 
cannot be determined unless the experiment is replicated (4). As this experiment consist 
of computer runs, a repetition of the entire set of conditions would result in exactly the 
same values containing the same errors due to the simplifying assumptions and convergence 
tolerances. 


RECOMMENDATIONS 

The L/D - Isp interaction problem can be circumvented by studying the ballistic and winged 
vehicles separately. An L27 orthogonal array has been selected for further study as the 
anticipated interactions between Isp and the remaining parameters can be evaluated as 
shown in Table IV. The same 27 trials will be repeated for both L/D configurations. This 
will require an additional 54 computer setups and enough computer runs to minimize 
vehicle mass at the specified parameter conditions. 


Table HI. ANOVA for Air Launch PLS 



L/D 

Iso 

■1 

Delav 

1M 

Error 

Results 







Trial no. 

SB 




m 

m 

r53 

Total AV 


1 

Ballistic 

LOX/LH2 

25 

0 

2 

1.5 

i i 

29,661 

730,965 

2 

Ballistic 

LOX/LH2 

3 

10 

4 

1.7 2 

2 

28.572 

661,753 

17 

Winged 

HTBP 

35 0 


6 

1.5 2 

3 

27,839 

3.502.700 

18 

Winged 

HTBP 

45 

10 

2 

1.7 3 

1 

27,542 

4,488,800 

Level 1 

14.113 

4.418 

10.456 

11.749 

11.857 

10.870 

1 ID vs IBS interactions 

Level 2 

19.592 

11.499 

10.953 

11.973 

11.106 

12.176 

Ball-LOx/LH2 

2.017 

Level 3 


17.788 

12.296 

9.983 

10.742 

10.659 

Ball-A50/N2O4 

5.088 

Totals 

33.705 

33.705 

33.705 

33.705 

33.705 

33.705 

Bail-HTBP 

7.007 








Wing-LOx/LH2 

2.400 

SS 

1.668 

14.915 

.302 

.396 

.108 

.225 

Wmg-A5(VN2/04 6.411 

V 

1 

2 

2 

2 

2 

2 

Wmg-HTBP 

10.781 

V 

1.668 

7.458 

.151 

.198 

.054 

.112 

SS 


17.604 








V 


2 

F 

4.57 

20.42 

.41 

34 

.15 

31 

V 


8.802 








F 


24.10 
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Table IV. L27 Orthogonal Array 


ORTHOGONAL ARRAY RECOMMENDED 
FOR FURTHER STUDY 

Iso T/W Alt Gamma Delay 

Column no. UQ. 

Trial no. 1 2 3 4 s 6 7 a 9 io ii 12 i3_ Ball'stig Winged 


1 

LOX/LH2 

1.5 

1 

1 

25K 

1 

1 

O' 

1 

1 

2 

1 

1 

2 

LOX/LH2 

1.5 

1 

1 

3SK 

2 

2 

10' 

2 

2 

4 

1 

1 

26 

HTBP 

1.9 

2 

1 

35K 

1 

3 

O' 

3 

2 

6 

2 

1 

27 

HTBP 

1.9 

2 

1 

45K 

2 

1 

20' 

1 

3 

2 

3 

2 



r 


An L27 (3 13 ) array is recommended for further study with the 
Ballistic and Winged studies to be done separately. The L27 
array was selected as it allows for the modeling of the 
interactions between Isp and each of the other parameters. 
The interactions between Isp and T/W will be modeled by 
columns 3 & 4, interactions between Isp and Altitude in 
columns 6 & 7, etc. This design should accurately evaluate the 
main effects (T/W, Altitude, Flight Path Angle, and Drop Time) 
as well as their interactions with Isp at a cost of requiring a 
total of 54 trials. 



If this new experiment design successfully obtains the information required by the study’s 
original statement of work, a total of 72 computer setup will have been required. Although 
the Taguchi Method is approaching the number of setups used in the traditional approach 
(90), a smaller number computer runs will be required using significantly less computer time 
than the traditional parametric study. 


CONCLUSIONS 

The application of Taguchi’s Design of Experiment Methods to improve the effectiveness 
of complex computer models is a new learning process subject to a learning curve. A 
common initial strategy requires the evaluation of a large number of parameters using a 
saturated array. This assists the analyst in determining which factors are relevant for further 
experiments. Knowledge of the system being modeled greatly reduces the initial effort 
required by improving the process of determining the proper parameters. The initial 

selection of the LI 8 matrix was made with the knowledge that interactions existed, but 
without anticipating their significance. The project is currently in the second phase with an 
experimental design better suited to the interactions found. 

As the study is not complete the anticipated results have not been achieved. The interim 
conclusions are as follows: 
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Anticipated results: 


* Improved systematic analysis process — While Taguchi’s design of experiment 
methods are clearly more systematic, this project has not demonstrated the methods 
effectiveness. 

* More information obtained at lower costs -- At this point in the project Taguchi 
Methods have not obtained the information required for the Air Launched PLS study. 

* More robust, cost effective designs — Sensitivity to noise and small variations in 
control parameters have not been addressed at this time. 

Actual results: 

* Considerable experience has been gained in the application of Taguchi Techniques 
in non-manufacturing environments. 

* The first point in the learning curve has been obtained. 
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